Some considerations regarding the use of multi-fidelity Kriging in the construction of surrogate models

Surrogate models or metamodels are commonly used to exploit expensive computational simulations within a design optimization framework. The application of multi-fidelity surrogate modeling approaches has recently been gaining ground due to the potential for further reductions in simulation effort over single fidelity approaches. However, given a black box problem when exactly should a designer select a multi-fidelity approach over a single fidelity approach and vice versa? Using a series of analytical test functions and engineering design examples from the literature, the following paper illustrates the potential pitfalls of choosing one technique over the other without a careful consideration of the optimization problem at hand. These examples are then used to define and validate a set of guidelines for the creation of a multi-fidelity Kriging model. The resulting guidelines state that the different fidelity functions should be well correlated, that the amount of low fidelity data in the model should be greater than the amount of high fidelity data and that more than 10\% and less than 80\% of the total simulation budget should be spent on low fidelity simulations in order for the resulting multi-fidelity model to perform better than the equivalent costing high fidelity model

Engineering design applications of surrogate-assisted optimization techniques

The construction of models aimed at learning the behaviour of a system whose responses to inputs are expensive to measure is a branch of statistical science that has been around for a very long time. Geostatistics has pioneered a drive over the last half century towards a better understanding of the accuracy of such ‘surrogate’ models of the expensive function. Of particular interest to us here are some of the even more recent advances related to exploiting such formulations in an optimization context. While the classic goal of the modelling process has been to achieve a uniform prediction accuracy across the domain, an economical optimization process may aim to bias the distribution of the learning budget towards promising basins of attraction. This can only happen, of course, at the expense of the global exploration of the space and thus finding the best balance may be viewed as an optimization problem in itself. We examine here a selection of the state of-the-art solutions to this type of balancing exercise through the prism of several simple, illustrative problems, followed by two ‘real world’ applications: the design of a regional airliner wing and the multi-objective search for a low environmental impact hous

An accelerated medial object transformation for whole engine optimisation

The following paper proposes an accelerated medial object transformation for the tip clearance optimisation of whole engine assemblies. A considerable reduction in medial object generation time has been achieved through two different mechanisms. Faces leading to unnecessary branches in the medial mesh are removed from the model and parallelisation of the medial object generation is improved through the subdivision of the original 3D CAD model. The time savings offered by these schemes are presented with respect to the generation of the medial objects of two complex gas turbine engine components. It is also demonstrated that the utilization of these techniques within a design optimisation may result in a considerable reduction in wall tim

The development of a hybridized particle swarm for kriging hyperparameter tuning

Optimizations involving high-fidelity simulations can become prohibitively expensive when an exhaustive search is employed. To remove this expense a surrogate model is often constructed. One of the most popular techniques for the construction of such a surrogate model is that of kriging. However, the construction of a kriging model requires the optimization of a multi-model likelihood function, the cost of which can approach that of the high-fidelity simulations upon which the model is based. The article describes the development of a hybridized particle swarm algorithm which aims to reduce the cost of this likelihood optimization by drawing on an efficient adjoint of the likelihood. This hybridized tuning strategy is compared to a number of other strategies with respect to the inverse design of an airfoil as well as the optimization of an airfoil for minimum drag at a fixed lif

Proper orthogonal decomposition & kriging strategies for design

The proliferation of surrogate modelling techniques have facilitated the application of expensive, high fidelity simulations within design optimisation. Taking considerably fewer function evaluations than direct global optimisation techniques, such as genetic algorithms, surrogate models attempt to construct a surrogate of an objective function from an initial sampling of the design space. These surrogates can then be explored andupdated in regions of interest.Kriging is a particularly popular method of constructing a surrogate model due to its ability to accurately represent complicated responses whilst providing an error estimate of the predictor. However, it can be prohibitively expensive to construct a kriging model at high dimensions with a large number of sample points due to the cost associated withthe maximum likelihood optimisation.The following thesis aims to address this by reducing the total likelihood optimisationcost through the application of an adjoint of the likelihood function within a hybridised optimisation algorithm and the development of a novel optimisation strategy employinga reparameterisation of the original design problem through proper orthogonal decomposition

On the potential of a multi-fidelity G-POD based approach for optimization & uncertainty quantification

Traditional multi-fidelity surrogate models require that the output of the low fidelity model be reasonably well correlated with the high fidelity model and will only predict scalar responses. The following paper explores the potential of a novel multi-fidelity surrogate modelling scheme employing Gappy Proper Orthogonal Decomposition (G-POD) which is demonstrated to accurately predict the response of the entire computational domain thus improving optimization and uncertainty quantification performance over both traditional single and multi-fidelity surrogate modelling scheme

Non-stationary kriging prediction of turbomachinery time variant responses

Surrogate models are usually employed in the representation of scalar values with little emphasis given to the modelling of time variant responses of scalar or vector quantities. The following paper aims to address this by presenting two non-stationary kriging based techniques which can be used to represent such quantities. These surrogate modelling strategies are demonstrated to accurately model the variation in turbomachinery component cycle performance with design changes thereby permitting future cross partner optimisations and trade-off studies

Performance of an ensemble of ordinary, universal, non-stationary and limit kriging predictors

The selection of stationary or non-stationary Kriging to create a surrogate model of a black box function requires a priori knowledge of the nature of response of the function as these techniques are better at representing some types of responses than others. While an adaptive technique has been previously proposed to adjust the level of stationarity within the surrogate model such a model can be prohibitively expensive to construct for high dimensional problems. An alternative approach is to employ a surrogate model constructed from an ensemble of stationary and non-stationary Kriging models. The following paper assesses the accuracy and optimization performance of such a modelling strategy using a number of analytical functions and engineering design problems

Applications of algorithmic differentiation within surrogate model generation

The construction of a surrogate model for the purposes of design optimisation often involves some form of sub-optimisation of the surrogate's controlling parameters. The construction of a kriging model, for example, can require a series of O(n^3) factorisations of the correlation matrix when performing the likelihood maximisation. Due to the smooth nature of the likelihood function, gradient information can be used to accelerate the likelihood optimisation when employed within a gradient enhanced global optimisation strategy. To this end a series of adjoints of the likelihood function of a variety of kriging based surrogate models are presented.An adjoint of the likelihood function derived via algorithmic differentiation is presented for traditional kriging. Recent extensions of this formulation to the likelihood functions for co-kriging and gradient enhanced kriging are also presented. Gradient enhanced kriging may be of particular interest to those wishing to employ derivative information from computational simulations, which itself may be a result of an algorithmic differentiation, within a design optimisation

Geometric filtration using proper orthogonal decomposition for aerodynamic design optimization

When carrying out design searches, traditional variable screening techniques can find it extremely difficult to distinguish between important and unimportant variables. This is particularly true when only a small number of simulations is combined with a parameterization which results in a large number of variables of seemingly equal importance. Here the authors present a variable reduction technique which employs proper orthogonal decomposition to filter out undesirable or badly performing geometries from an optimization process. Unlike traditional screening techniques, the presented method operates at the geometric level instead of the variable level. The filtering process uses the designs which result from a geometry parameterization instead of the variables which control the parameterization. The method is shown to perform well in the optimization of a two dimensional airfoil for the minimization of drag to lift ratio, producing designs better than those resulting from traditional kriging based surrogate model optimization and with a significant reduction in surrogate tuning cos